Multi-index antithetic stochastic gradient algorithm

Abstract Stochastic Gradient Algorithms (SGAs) are ubiquitous in computational statistics, machine learning and optimisation. Recent years have brought an influx of interest SGAs, the non-asymptotic analysis their bias is by now well-developed. However, relatively little known about optimal choice random approximation (e.g mini-batching) gradient SGAs as this relies on variance problem specific. While there been numerous attempts to reduce these typically exploit a particular structure sampled distribution requiring priori knowledge its density’s mode. In paper, we construct Multi-index Antithetic Algorithm (MASGA) whose implementation independent target measure. Our rigorous theoretical demonstrates that for log-concave targets, MASGA achieves performance par with Monte Carlo estimators access unbiased samples from interest. other words, estimator mean square error-computational cost perspective within class estimators. To illustrate robustness our approach, implement also some simple non-log-concave numerical examples, however, without providing guarantees algorithm’s such settings.